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Abstract 


In civil aviation baggage consignment and sorting, it is a necessary func- 
tion to automatically detect whether a pallet is added to the self-dropped 
baggage. However, the pallet is largely obscured by the embedded baggage, 
and the pallet detection becomes a challenging problem. For this issue, a 
fast detection algorithm for embedded baggage pallets based on multi-layer 
skeleton model registration is proposed. In order to describe the characteris- 
tics of the pallet, the point cloud skeleton model and the point-line model are 
constructed by the 3D point cloud model. During online detection, firstly, 
the designed feature description and banded point extraction method is used 
to grab the border point cloud, and the proposed point-line potential energy 
adaptive iterative algorithm is used to registration the point-line model and 
horizontal border point set. Then, point cloud iterative nearest point reg- 
istration based on random sampling consistency is used to achieve accurate 
registration and pose calculation. As a result, the possibility of the existence 
of the pallet is judged. The effectiveness of the algorithm is verified by a 
variety of actual pallet detection experiments. A variety of typical compar- 
ative experimental results show that when the pallet point cloud is missing 
within 70%, the algorithm can still maintain the accuracy of 94%. At the 
same time, the speed exceeds The typical algorithm is more than 6 times. 


Keywords: 3D object detection, baggage pallet, multi-layer skeleton model, 
point clouds registration 
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1. Introduction 


The self-service baggage check system, which frees airport attendants 
from heavy duty service, significantly improves the efficiency and quality of 
airport operations. The system needs to automatically detect the shape of 
the baggage delivered by passengers themselves, including the number, size, 
shape and type, as well as whether the soft package is equipped with pallets, 
and the number of baggage pieces in pallets. Automatic detection of whether 
the soft baggage delivered by passengers is equipped with pallets is the key 
to ensuring the safety of passenger baggage and baggage sorting system. 

The detection of such objects as baggage pallets generally uses industrial 
cameras or laser radar to obtain surface images or three-dimensional point 
clouds and the object recognition is carried out by extracting and matching 
color, texture and structural characteristics|1]. In the open baggage drop 
area, the reliability of the pallet detection method based on images anal- 
ysis will be reduced due to the complex ambient light, the interference of 
the luggage surface texture, and the uncertainty of the drop position.In con- 
trast, the object detection method based on point clouds is less affected by 
environmental factors. Therefore, it is a more reliable solution to use the 
Three-dimensional(3D) object detection method based on point cloud anal- 
ysis to complete the pallet detection of embedded baggage. 

3D object detection based on point cloud generally first extracts the local 
or global features of the object from the point cloud to matches the scene 
features, and judges whether the object exists in the scene according to the 
matching evaluation results. Then, the precise pose of the target is estimated 
through point cloud registration[2, 3, 4jor pose clustering|[5, 6, 7]. However, 
the air baggage pallet is embedded and occluded by the baggage during 
baggage self-service shipment, and only the border or part of the border 
is exposed, which makes it difficult for the traditional 3D object detection 
method to work together. 

For this issue,this paper researched a 3D detection method for embedded 
occluded objects with partial missing point cloud.Based on the known struc- 
ture of the pallet and considering the known rectangular characteristics and 
horizontal upward placement characteristics, a specific three-layer skeleton 
model and adopt the model registration method of step-by-step refinement is 
designed to achieve fast and accurate detection of baggage pallets. The main 
contributions of this study are as follows: 

(1)The established multi-layer skeleton model can well describe the mor- 
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phological characteristics of the baggage pallet. 

(2)Designed banded point cloud descriptions and extraction methods can 
effectively extract the banded borders of the point cloud from the target 
scene, realize the separation of pallets and baggage. 

(3)The proposed point-line model registration algorithm based on iter- 
ative gravitational potential energy can realize the mapping and matching 
of the horizontal point set to the rectangle, complete the registration of the 
pallet point-line model and the horizontal banded set. 

(4)The proposed point cloud registration algorithm based on multi-layer 
skeleton model can realize fast and accurate detection of baggage pallets 
under the condition of large area missing point cloud. 

The main structure of this paper is as follows. Section 2 briefly introduces 
the related work. Section 3 Introduce the principle of multi-layer skeleton 
model registration method. Section 4 gives the analysis and the experiment. 
Section 5 concludes the work of this paper. 


2. Related Work 


Many different methods have been proposed for 3D object detection and 
recognition based on point cloud[8, 9], such as 3D feature descriptors based 
methods, graph matching based methods and machine learning based meth- 
ods. Usually,3D feature descriptors based methods,extract local key point 
neighborhood features or global statistical features of objects and scenes, and 
complete object discrimination by feature matching[10, 11, 12, 13]. Graph 
matching based methods|14] decomposes point cloud data into basic shapes, 
uses the adjacent relationship between shapes to represent three-dimensional 
objects, and obtains the position of the object through matching. Machine 
learning based{15] methods use samples to train classifiers to complete object 
detection and classification. Among them, the 3D object detection method 
based on deep learning/16, 17], through a large number of labeled samples, 
training multi-layer deep neural networks, can obtain good generalization 
performance, get a higher detection accuracy, such as PointNet/18], LIDAR 
R-CNN[19],etc. 

However, when the object is occluded, the accuracy of traditional meth- 
ods decreases significantly. Registration with local descriptors such as Fast 
Point Feature Histograms(FPFH)[10], Signature of Histogram of Orienta- 
tion(SHOT) [11] can overcome the influence of small occlusion and back- 
ground interference.Point Pair Feature(PPF)|[7] and its improvement([20] per- 
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forms object recognition and pose estimation through the correspondence be- 
tween point pairs, which is still effective when the target is partially occluded. 
Detection method based on three-dimensional Hough voting|21],Rotational 
Subgroup Voting|22] and clutter-oriented detection method|23] further solved 
the problem of target occlusion and achieved good results on various data 
sets.When the object occlusion is serious and the proportion of incomplete 
point cloud exceeds 50%, the detection of three-dimensional object becomes 
more difficult, and more target feature information or multi-sensor informa- 
tion fusion information is needed to complete the detection. 

In summary, to overcome the impact of incomplete point cloud on 3D ob- 
ject detection, many scholars have done a lot of research. The pallet loaded 
with baggage only shows the border, sometimes the degree of occlusion can 
reach more than 70%, and the accuracy of 3D target point cloud obtained 
by the sensor is not high due to the cost limitation, resulting in the per- 
formance of the traditional 3D target detection method can not meet the 
practical requirements.In contrast, this paper proposes a multi-layer skele- 
ton model description and registration method to effectively overcome the 
baggage occlusion of the pallet, which meets the actual needs of self-service 
luggage check-in systems. 


3. Methods 


The fundamental reason why traditional point cloud target detection 
methods are prone to mismatch under occlusion conditions is that the fea- 
tures of occluded targets are not obvious. If the object model is known, the 
specific model of the object is established according to the structure or local 
characteristics of the object, and the model registration is adopted, which can 
improve the success rate of target detection|24]. Point cloud skeleton model 
has the characteristics of simple and obvious topological structure, and the 
use of skeleton model registration can effectively utilize the structural fea- 
tures of the object, which helps to overcome the problem of misregistration 
caused by occlusion.The algorithm flow as shown in Fig.1. 


3.1. Pallet Modeling 


The shape and size of the air baggage pallets in each airport are not 
significantly different and the specifications of baggage pallets in the same 
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Figure 1: Algorithm flowchart 


terminal are always same. In the same airport application scenario, the 
three-layer skeleton model of pallet can be established offline. 

Fig.2(a) shows one type of baggage pallet in the airport.Firstly, scanning 
the empty pallet by three-dimensional laser and ignoring the bottom, the 
point clouds modelMzis obtained, as shown in Fig.2(b). 

Direct use of point cloud model registration and detection can easily lead 
to mismatches and errors when pallets are obscured by loaded luggage. The 
pallet border, especially the upper surface of the border, is basically vis- 
ible in the scene cloud, and the border can fully show the morphological 
characteristics of the pallet. Therefore, the L1 center skeleton extraction 
algorithm[25] is used to establish the border skeleton model. The skeleton 
modeling process as follows: (1)Manually select the upper surface points 
from the point cloud model; (2)Randomly select a certain amount of sam- 
pling points to determine the appropriate neighborhood and construct the 
initial skeleton model; (3) Expanding the neighborhood scope, using local 
L1 median shrinkage sampling points, and iteratively constructing different 
regional skeletons; (4)After the skeleton is smoothed and concentrated, the 
border-skeleton model Mg of the pallet is obtained,as shown in Fig.2(c). 

It is difficult to extract the complete pallet border when the frame of bag- 
gage shielding part is put in, and the skeleton model is also prone to produce 
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matching errors.The horizontal projection of the pallet frame is a rectangle, 
which can well describe the geometric information of the pallet. Therefore,the 
border skeleton model is mapped to a horizontal plane, forming a plane 
point-line model Mz, which is composed of four corner point, can be said for 
Mgr = {(xi, yi) li = 1, 2,3, 4}, as shown in Fig.2(d). 


(a) 


Figure 2: baggage pallet and three-skeleton model;(a)is a baggage pallet,(b)is the point 
cloud model, (c)is the border-skeleton model,(d)is the point-line model 


3.2. Banded feature description and pallet border point cloud extraction 


During online detection, due to the small number of occluded pallet 
points, it is difficult to maintain a high success rate by directly registering 
the scene point cloud with point cloud model of the pallet. If the baggage 
and pallet point cloud can be separated, the misregistration problem caused 
by embedded baggage can be effectively overcome by only using pallet point 
cloud to register with pallet model. Combined with the strip distribution 
characteristics and horizontal placement characteristics of pallet, a strip fea- 
ture description and extraction method is designed to grab the upper surface 
of the border points and realize the separation of baggage and pallet point 
cloud. 

(1)Banded feature description 

According to the relative position distribution of point clouds, three- 
dimensional scanning point clouds can be divided into two categories:non- 
banded points and banded points, as shown in Fig.3. The non-banded points 
are located at the edge and inside of the large-scale point cloud area, which 
is generally the point cloud on the upper surface of the baggage. The banded 
points are distributed in a linear band, which is generally the point cloud of 
the pallet border. 
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Figure 3: Points with two relative position distributions 


In order to distinguish the above non-banded points and banded points, 
the covarance matrix of neighborhood points is defined to describe the rela- 
tive position distribution characteristics within the point cloud. For the point 
C= (a, z)" the X, Y coordinate plane is horizontal, Z coordinate axis di- 
rection is vertical upward. In the k-neighborhood point set of q, Q; contains 
n field points, so the covariance matrix of Qx can be defined by Formula(1). 


n 


Cla) =X (ai-Da-9" (1) 


i=l 


The maximum eigenvalue of C(q) is Amax, and its eigenvalue isA;,i = 1,2. 
For point q, if Amax >> Ai, then q is a banded point. For non-banded points, 
there is little difference between the eigenvalues. 

(2)Pallet border point cloud extraction 

The pallet is horizontally placed in the baggage channel, and there is no 
pallet suspension or inversion. The height range of point cloud in the border 
is known, so the banded point cloud extraction process of pallet border is as 
follows: 

1)According to the known height range of the pallet border, select the 
pallet border candidate point cloud Qsec from the scene cloud Qp. 

2)For a planar mapping point q = {x,y} of any point q in Qsec, a set of n 
neighborhood points is obtained by searching the points in the neighborhood 
of k, denoted by:Q mat = {qi = (mu) = 12; eae 
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3)The covariance matrix C (7) of Q mat is calculated by (2). 


(=E (u-i) i) 2) 


C (7) is a two-dimensional square matrix, which can get two eigenvalues Ay, 
Az, introduced coefficient l, calculated by (3). 


l = max (|A1/Ag| , |A2/A1]) (3) 


Set a threshold ¢(4by experience ) when the corresponding point is a banded 
point. 

4) Repeat the above steps, traverse Qsec, get pallet banded border point 
cloud Qs. After horizontal projection, the horizontal border point set Qz is 
obtained. 


3.8. Three-layer skeleton model registration 

After getting the three-layer skeleton model and the banded points on the 
pallet border of the scene, online matching is needed to evaluate whether the 
scene contains the pallet and mark the exact pose of the pallet. Therefore, 
according to the different structure of the three-layer skeleton model , a multi- 
level model registration method is designed, and the pallet model registration 
is completed in the plane, border skeleton and three-dimensional point clouds 
level. 

(1) Registration of planar point-line model 

The horizontal border point set Qz is a discrete lattice, and the planar 
point-line model Mz is the fitted rectangular boundary. The registration 
process is actually the mapping transformation process from the point set to 
the rectangle. Therefore, an evaluation index called ’point-line gravitational 
potential energy’ is defined to measure the correctness of the mapping from 
point set to rectangle, and an adaptive iterative registration algorithm is 
designed based on this. 

As shown in Fig.4, a; is a point in Qz, b; is the nearest point to a; on Mz, 
O is the rotation center of Mz, c; is the foot of a;b; on O line. The Mz is 
continuously rotated and shifted by the traction of point a;, and the gravity 
T; and torque R; are calculated by formulas (4) and (5). 


— 


biai G 
= *§ = (4) 
biai biai 
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Figure 4: Point-line registration model 


Where G is a constant, the average traction Ts and torque Rs of the planar 
point-line model Mz of all points in Qz are obtained by formulas (6) and (7). 


— 
biai G 
T; = U4 e (6) 
Res SR (7) 
S né 4 


where n is the number of Q; points. Under the traction of horizontal frame 

banded lattice Qz, Mz gradually approaches to Qz. In the transformation 

process, the coincidence degree of the two can be evaluated by potential 
energy E, and E is defined by Formula (8). 
7 G 

Paos 8 

= min {l;; |j = 1,2, 3,4} (8) 


where l;; is the distance from any point in Qz to the rectangular four edges of 
the planar point-line model. The purpose of registration is to find a suitable 
transformation, so that the potential energy between the transformed plane 
point-line model and the horizontal border point set is the smallest, then the 
objective function expression is formula(9). 


{ (RT) = aming (QL, Mi (R, T)) (9) 
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Where R is a rotation matrix and T is a translation vector. In the reg- 
istration process, under the constraints of the potential energy E and the 
objective function, Mz is iteratively transformed. During the iteration, the 
step length of the transformation is adaptively adjusted by gravity. The 
relationship between translation Ti and rotation angle 6; of the ith transfor- 
mation and gravitation and torque is shown in formula(10)(11). 


T; = steprTs; (10) 


0; = stepyRs, (11) 


Among them, stepr and stepo is a constant, Ts, and Rg, is the average 
traction and torque of the ith transformation, and the corresponding trans- 
formation relationship R and T are solved by formula(12). 


cosĝ; —sin§; 0 z bee 
Mr’ = RMr; +T = | sinð; cosð; 0 y | F| ty (12) 
0 0 1 0 0 


In the formula, Mz’ and Mz; are the plane point-line models before and after 
the ith transformation, ts, and t,, are the components of Rs, in x and y 
respectively. Recalculate the potential energy after each transformation and 
make the next iteration. The iteration is terminated when the number of 
iterations reaches the upper limit or the potential energy stabilizes. 

In the registration process, the absolute value of potential energy | E| can 
reflect the coincidence degree between the horizontal border point set and 
the pallet point-line model. The higher the coincidence degree, the greater 
the probability of containing the pallet. Set a threshold Emin( Emin > 0, the 
empirical value is 150 ), after the iteration is completed, if |E| > Emin, the 
existence of pallet can be determined. 

(2)Registration of border-skeleton model 

Horizontal border point set loses some three-dimensional spatial charac- 
teristics, and the vertical position and orientation of the pallet need to be 
corrected at the border skeleton level. There are differences between the po- 
sition and orientation of the pallet banded border point cloud Qs and the 
border-skeleton model Mg , and the shape is also different. RANSAC can be 
used to select the corresponding point pairs to complete the registration. 
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Firstly, selected the nearest neighbor points as the corresponding point 
pairs from the pallet banded border point cloud and skeleton model after 
initial transformation by applying point-line model registration. The process 
is as follows: 

For the point m; in Ms, the nearest point q; is searched from Qg. At the 
same time, for the point q; in Qs, the nearest point m; is searched from Ms. 
If m;, qj are the closest corresponding points to each other and the distance 
is less than the set threshold, then (m;,q;) is a pair of corresponding points. 

After the selection of corresponding points, randomly select n groups of 
corresponding points, estimate transformation, and calculate the distance 
d = ||R’m+ T’ — q|| after transformation for remaining points.If d is less 
than the given threshold, the point pair is called as a pair of inner points. 
Set the maximum number of iterations N, counted the number of interior 
points corresponding to each transformation until the maximum number of 
iterations is reached. After the iteration, the transformation with the largest 
number of inner points is the best transformation of the model, which is the 
coarse position of the pallet. 

(3) Registration of point clouds model 

The point-line model registration and the border-skeleton model regis- 
tration are matched with the extracted banded border at the plane and 
border-skeleton levels. The above registration only applies the point cloud 
of the border on the pallet, and the point cloud of other parts of the pallet 
is not involved in the calculation. In order to calculate the pallet pose more 
accurately and detect and extract all the pallet point cloud, it is necessary 
to use the 3D point cloud model of the pallet to register with the field point 
cloud. 

ICP|2] is a classical algorithm for point cloud registration. The principle 
is to find the nearest point in the target point cloud and source point cloud 
according to certain constraints, and calculate the optimal transformation 
parameters R and T through iterative matching. However, the civil aviation 
baggage pallet is embedded and occluded and the weight of pallet in point 
clouds is low. Moreover, the characteristics of baggage surface and pallet side 
are similar, so the direct ICP registration is prone to mismatch. In order to 
prevent ICP from falling into local optimum, a certain distance threshold is 
set under the constraint of initial pose of skeleton registration. By extracting 
the nearest point, the overlapping area of two point clouds is obtained. The 
ICP algorithm is applied to the overlapping area to obtain the accurate pose 
of the pallet. 
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4. results of pallet detection 


The experimental hardware platform adopts Intel Core i5-7300HQ CPU 
and a computer with 8 GB memory. Under the Windows 10 operating system 
platform, the algorithm is realized based on OpenCV and PCL development 
library using the software development tool Visual2019. The experimental 
platform is shown in Fig.5. Beiyang URG-04LX-UG01 is used as the 3D laser 
scanner. The scanning area of the sensor is 20-5600 mm, and the accuracy 
is +30 mm. The experimental data are 490 sample data randomly selected 
from 5 million data of an airport in Guangzhou from 2019 to 2021. 


Figure 5: Self-service baggage consignment experiment platform 


Select typical samples under various conditions, as shown in Table.1. 
Sample 1 is a typical pallet sample embedded in baggage and without border 
cover, sample 2 is an empty pallet, and samples 3 to 6 are pallet samples with 
different integrity. In the experiment, in order to show the registration effect 
of point cloud more clearly, the point cloud is used to represent the pallet 
model, and the reconstructed mesh model is used to represent the scene point 
cloud of the test field. 


4.1. Pallet border point cloud extraction 


The effectiveness of border point cloud extraction algorithm is verified 
by selecting the typical sample 1 of unshielded pallet border. The banded 
boundary of sample 1 is extracted by using the banded point cloud extraction 
algorithm, and the results are shown in Fig.6(a), which are projected onto the 
horizontal plane to obtain the horizontal boundary point of pallet, as shown 
in Fig.6(b). It can be seen that the banded point cloud mainly includes the 
side points of the border on the pallet, but the corner points and the adjacent 
baggage cannot be correctly extracted. The reason is that the distribution of 
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Table 1: Results of different algorithms. 


Data sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 
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A D 


0% 16.8% 35.0% 62.4% 73.8% 


point cloud 


Defect ratio 
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corner points and luggage adjacent points is relatively concentrated, which 
is mistaken for non-banded points. In general, the proposed banded point 
cloud extraction algorithm can extract the pallet border point cloud from 
the scene point cloud, and realize the separation of baggage and pallet point 


a ~ 
N? 


Figure 6: Results of pallet border point cloud extraction;(a)is the pallet banded border 
point cloud (b)is the horizontal border point set 


4.2. Three-layer skeleton model registration 


The effectiveness of the three-layer skeleton model registration algorithm 
is verified by selecting the typical sample 1 of unshielded pallett border. 
(1)Registration of planar point-line model 
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In order to verify the effectiveness of the point-line gravitational potential 
energy registration algorithm, the registration experiment is carried out by 
using the pallet plane point-line model and the horizontal boundary point 
set extracted from sample 1. Before registration, the position of the pallet 
plane point-line model is quite different from the horizontal boundary point 
set of the scene to be tested, as shown in Figure.7(a). After registration, the 
two are combined, as shown in Fig.7(b). 


(a) (b) 


Figure 7: Results of planar point-line model Registration; (a)is before registration,(b)is 
after registration 


The process of potential energy change is shown in Fig.8. Before regis- 
tration, the absolute value of potential energy is relatively small. With the 
increase of iterations, the point-line model of pallet plane is continuously 
transformed under the traction of the horizontal boundary point set, and 
the absolute value of potential energy gradually increases. When the two 
coincide, the absolute value of potential energy tends to be maximized and 
stabilized. After registration, the absolute value of potential energy is greater 
than the set threshold, and the existence of pallet is determined. It can be 
seen that the proposed adaptive iterative registration algorithm of point-line 
gravitational potential energy can effectively evaluate the possibility of pal- 
let existence in the scene to be tested, and complete the matching at the 
horizontal plane, providing the initial pose for subsequent registration. 

(2)Registration of border-skeleton model 

In order to verify the effectiveness of the border-skeleton model registra- 
tion algorithm, the pallet border skeleton model and sample 1 band border 
point cloud registration are used. Before registration, under the constraint 
of initial pose, the pallet border-skeleton model and the banded border point 
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Figure 8: Change of potential energy in point-line model registration 


cloud overlap on the horizontal plane, but there are still deviations in the 
vertical direction, as shown in Fig.9(a). After registration, the two also over- 
lap in the vertical direction, as shown in Fig.9(b). It can be seen that the 
registration of the border-skeleton model can complete the coarse registra- 
tion of the pallet border-skeleton model and the banded border point cloud 
in the vertical direction. 


Figure 9: Results of border-skeleton model Registration;(a)is Before registration,(b)is Af 
ter registration 


(3) Registration of point clouds model 

After the registration of the border-skeleton model, the accurate pose 
of the pallet is determined by point cloud registration. The initial pose of 
the point-line model registration and the border skeleton model registration 
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are used to register the point cloud model of the coarse pose transforma- 
tion pallet. After extracting the overlapping area, the ICP algorithm was 
used to fine-tune point cloud sample.1. Before the transformation, the posi- 
tion relationship between the pallet point cloud model and the scene point 
cloud to be measured is shown in Fig.10(a), and the position relationship 
after registration is shown in Fig.10(b). It can be seen that after multi-layer 
transformation and registration, the pallet point cloud model is basically in 
the same position with the pallet point cloud of the scene to be tested, but 
there are still slight differences. The reason is that the point cloud model es- 
tablished offline is different from the pallet point cloud of the actual scene to 
be tested, resulting in incomplete overlap. The overall experimental results 
show that the proposed three-layer skeleton model registration method can 
effectively detect the baggage pallet and mark the precise pose of the pallet. 


(b) 


Figure 10: Results of point clouds model Registration;(a)is before registration, (b)is after 
registration 


4.3. results of pallet detection 


Typical samples 1 to 6 were selected for pallet detection experiment to 
evaluate the influence of pallet point cloud defect ratio on the proposed 
method, and compared with other typical point cloud registration algorithms 
ICP, SHOT-ICP and FPFH. As shown in Table.2, the experimental results 
show that each method can achieve better registration effect in samples 1 and 
2 which are without pallet point missing. However, when the pallet point 
cloud is incomplete, the typical algorithm is no longer suitable. In sample 
3, ICP and FPFH have failed when a small part of the pallet point cloud 
is incomplete. When SHOT provides a good initial position for ICP, it can 
be successfully registered. When the pallet defect reaches more than 40%, 
SHOT-ICP cannot be effectively detected, and the detection results of the 
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proposed method remain stable. Under the condition of large-scale point 
cloud missing, the registration can still be successful. 

In order to quantitatively analyze the registration accuracy, RMSE [26] 
is used as the evaluation index, as shown in Formula (13). 


(13) 
where n is the number of pallet points in the scene to be tested, m is the 
number of pallet point cloud model points, and the registration accuracy 
is shown in Table.3. It can be seen from the table that when the pallet is 
complete, the typical algorithm SHOT-ICP has the best registration effect, 
and the accuracy of the proposed algorithm is not much different. When the 
pallet point cloud is missing, the accuracy of the proposed method is signifi- 
cantly better than other methods. The above results show that the proposed 
method ensures the accuracy of detection when the pallet is complete, and 
has obvious advantages for the incomplete point cloud. 

The practicability of the algorithm is verified by statistical analysis of 
point cloud samples with different integrity and without pallets. The results 
are shown in Table.4.It can be seen that all the 268 groups of pallet-free 
samples were tested correctly. In the 222 groups of pallet-containing samples, 
all the samples with pallet defect ratio less than 50% were tested correctly. In 
the 17 groups of samples with pallet defect ratio of 50%—-70%, only one group 
of test errors were detected, and the detection accuracy was 94.1%. When 
the incomplete ratio of pallet point cloud is greater than 70%, the detection 
accuracy will be greatly reduced, and only one group of 13 pallet integrity 
samples is correct. The reason is that the banded point cloud of the pallet 
border is too sparse, resulting in the absolute value of the potential energy 
of the point-line model is small, and is misjudged as no pallet. Overall, the 
detection accuracy of the proposed algorithm is more than 94%, which has 
obvious practical value. 


4.4. Time complexity analysis 

In order to verify the rapidity of the method, the registration time of a 
single typical point cloud sample is compared with other algorithms. The 
results are shown in Table.5. It can be seen from Table 5 that the average 
time of ICP registration is about 2.41 s, the average time of SHOT-ICP is 
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Table 3: Accuracy comparison of typical registration algorithms. 


Data RMSE/mm 
ICP FPFH SHOT-ICP Proposed 
sample 1 13.3 14.8 11.1 11.9 
sample 2 6.5 5.8 59 8.9 
sample 3 69.1 91.3 12.4 12.2 
sample 4 31.9 32.1 32.1 14.5 
sample 5 41.9 116.6 110.7 17.2 
sample 6 31.5 31.6 33.7 18.5 


Table 4: Statistical experimental results. 


Non-pallet Containing pallet Point Cloud Defective pallet Sample (Defective Ratio) 
sample types 


sample samples 


<10%  10%-30%  30%-50% 50%-70% >70% 
number of samples 268 222 129 39 24 17 13 
Accuracy 100% 94.1% 100% 100% 100% 94.1% 7.7% 


about 23.97 s, and the average time of FPFH is about 42.67 s. The average 
time required for the proposed algorithm is less than 0.4 s, which is obvi- 
ously due to other algorithms. The reason is that the proposed three-layer 
skeleton model registration method only needs to calculate the distribution 
of the neighborhood position of the point cloud when extracting the pallet 
features, which reduces the calculation consumption and greatly reduces the 
calculation amount through multi-level local point registration. 


Table 5: Time of different registration algorithms. 


Data Time/mm 
ICP FPFH SHOT-ICP Proposed 
sample 1 1.48 15.31 32.96 0.44 
sample 2 2.36 28.94 48.69 0.46 
sample 3 2.87 28.46 34.33 0.38 
sample 4 2.26 29.52 41.77 0.34 
sample 5 2.85 26.34 47.27 0.29 
sample 6 2.63 25.27 23.31 0.23 
average time 2.41 23.97 42.67 0.35 
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5. Conclusion 


In this paper, a 3D object detection method based on three-layer skeleton 
model is studied to solve the problem of fast and accurate detection of pallets 
when large area occlusion of baggage is embedded in self-service baggage 
consignment. 

The built skeleton model can better describe the three-dimensional char- 
acteristics of baggage pallet. Through the gradual refinement registration of 
the three-layer skeleton model of plane, skeleton and point cloud, the accu- 
racy of pallet detection can be more than 94% under the condition of 70% 
point cloud residue. The feature description and extraction method of strip 
point cloud and the adaptive iterative registration algorithm based on point 
line gravitational potential energy can accurately extract the border point 
cloud of the pallet, avoid the influence of baggage point cloud on the pal- 
let model registration, and effectively improve the detection speed, which is 
more than 6 times higher than the typical three-dimensional target detection 
algorithm. 

The method detects the pallet based on the registration degree of the 
known three-dimensional structural skeleton model. When the pallet type 
is known and the type is small, it can be applied in civil aviation airports. 
However, for the occlusion detection of other unknown structures, it has 
limitations and needs further improvement and optimization. 


Acknowledgement 


This work was supported by the Tianjin Education Commission(2019KJ117). 


References 


[1] M. Kim, S. Byun, J. Kim, A monocular vision based technique for esti- 
mating direction of 3d parallel lines and its application to measurement 
of pallets, Journal of Korea Multimedia Society 21 (2018) 1254-1262. 
doi:10.9717/KMMS .2018.21.11.1254. 


[2] P. J. Besl, H. D. Mckay, A method for registration of 3-d shapes, IEEE 
Transactions on Pattern Analysis and Machine Intelligence 14 (1992) 
239-256. doi:10.1109/34.121791. 


20 


202204.00127v1 


chinaXiv 


[3] 


[10] 


B. Gka, Z. A. Qi, A. Jw, Z. A. Han, Pose estimation of 
a non-cooperative spacecraft without the detection and recognition 
of point cloud features, Acta Astronautica 179 (2021) 569-580. 
doi:10.1016/j.actaastro.2020.11.013. 


Z. Yao, Q. Zhao, X. Li, Q. Bi, Point cloud registration algorithm 
based on curvature feature similarity, Measurement 177 (2021) 109274. 
doi:https://doi.org/10.1016/j .measurement . 2021. 109274. 


C. Choi, Y. Taguchi, O. Tuzel, M. Y. Liu, S. Ramalingam, Voting-based 
pose estimation for robotic assembly using a 3d sensor, in: Proceedings 
- IEEE International Conference on Robotics and Automation, 2013, 
pp. 1724-1731. doi:10.1109/ICRA.2012.6225371. 


J. Guo, X. Xing, W. Quan, D.-M. Yan, Q. Gu, Y. Liu, X. Zhang, Ef- 
ficient center voting for object detection and 6d pose estimation in 3d 
point cloud, IEEE Transactions on Image Processing 30 (2021) 5072- 
5084. doi:10.1109/TIP. 2021 .3078109. 


B. Drost, M. Ulrich, N. Navab, S. Ilic, Model globally, match locally: 
Efficient and robust 3d object recognition, in: 2010 IEEE computer 
society conference on computer vision and pattern recognition, Ieee, 
2010, pp. 998-1005. doi:10.1109/CVPR.2010.5540108. 


G. Du, K. Wang, S. Lian, Vision-based robotic grasping from 
object localization, pose estimation, grasp detection to motion 
planning: A review, arXiv preprint arXiv:1905.06658 (2019). 
doi:https://doi.org/10.48550/arXiv.1905.06658 


C. Sahin, G. Garcia-Hernando, J. Sock, T.-K. Kim, A review on 
object pose recovery: from 3d bounding box detectors to full 6d 
pose estimators, Image and Vision Computing 96 (2020) 103898. 
doi:https://doi.org/10.1016/j.imavis.2020. 103898. 


R. B. Rusu, N. Blodow, M. Beetz, Fast point feature his- 
tograms (fpfh) for 3d registration, in: 2009 IEEE International 
Conference on Robotics and Automation, 2009, pp. 3212-3217. 
doi:10.1109/ROBOT . 2009. 5152473. 


21 


202204.00127v1 


chinaXiv 


[11] 


[12] 


[13] 


[14] 


[15] 


[16] 


[17] 


[18] 


[19] 


S. Salti, F. Tombari, L. D. Stefano, Shot: Unique signatures of his- 
tograms for surface and texture description, Computer Vision & Image 
Understanding 125 (2014) 251-264. doi:10.1016/j.cviu.2014.04.011. 


J. Ligon, D. Bein, P. Ly, B. Onesto, 3d point cloud processing using spin 
images for object detection, in: 2018 IEEE 8th Annual Computing and 
Communication Workshop and Conference (CCWC), 2018, pp. 731-736. 
doi:10.1109/CCWC . 2018 .8301688. 


Y. Zhang, C. Li, B. Guo, C. Guo, S. Zhang, Kdd: A kernel density 
based descriptor for 3d point clouds, Pattern Recognition 111 (2021) 
107691. doi:https://doi.org/10.1016/j .patcog.2020.107691. 


W. Hao, Y. Wang, Structure-based object detection from 
scene point clouds, Neurocomputing 191 (2016) 148-160. 
doi:https://doi.org/10.1016/j .neucom.2015.12.101. 


H. Wang, W. Cheng, H. Luo, L. Peng, Y. Chen, J. Li, 3d 
point cloud object detection based on supervoxel neighborhood with 
hough forest framework, IEEE Journal of Selected Topics in Ap- 
plied Earth Observations & Remote Sensing 8 (2017) 1570-1581. 
doi:10.1109/JSTARS . 2015. 2394803. 


Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu, M. Bennamoun, Deep 
learning for 3d point clouds: A survey, IEEE Transactions on 
Pattern Analysis and Machine Intelligence 43 (2021) 4338-4364. 
doi:10.1109/TPAMI . 2020 . 3005434. 


Y. Cui, R. Chen, W. Chu, L. Chen, D. Tian, Y. Li, D. Cao, Deep 
learning for image and point cloud fusion in autonomous driving: A 
review, IEEE Transactions on Intelligent Transportation Systems 23 
(2022) 722-739. doi:10.1109/TITS . 2020 . 3023541. 


R. Q. Charles, H. Su, M. Kaichun, L. J. Guibas, Pointnet: Deep learning 
on point sets for 3d classification and segmentation, in: 2017 IEEE 
Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 
pp. 77-85. doi:10.1109/CVPR. 2017.16. 


Z. Li, F. Wang, N. Wang, Lidar r-cnn: An efficient and univer- 
sal 3d object detector, in: 2021 IEEE/CVF Conference on Com- 


22 


202204.00127v1 


chinaXiv 


[20] 


[21] 


[23] 


[24] 


[25] 


[26] 


puter Vision and Pattern Recognition (CVPR), 2021, pp. 7542-7551. 
doi:10.1109/CVPR46437 . 2021 .00746 


D. Li, H. Wang, N. Liu, X. Wang, J. Xu, 3d object recog- 
nition and pose estimation from point cloud using stably ob- 
served point pair feature, IEEE Access 8 (2020) 44335-44345. 
doi:10.1109/ACCESS . 2020 . 2978255 


F. Tombari, L. Di Stefano, Object recognition in 3d scenes with 
occlusions and clutter by hough voting, in: 2010 Fourth Pacific- 
Rim Symposium on Image and Video Technology, 2010, pp. 349-355. 
doi:10.1109/PSIVT.2010.65. 


A. G. Buch, L. Kiforenko, D. Kraft, Rotational subgroup voting and 
pose clustering for robust 3d object recognition, in: 2017 IEEE Inter- 
national Conference on Computer Vision (ICCV), 2017, pp. 4137-4145. 
doi:10.1109/ICCV. 2017. 443. 


W. Guo, W. Hu, C. Liu, T. Lu, 3d object recognition 
from cluttered and occluded scenes with a compact local fea- 
ture, Machine vision and applications 30 (2019) 763-783. 
doi:https://doi.org/10.1007/s00138-019-01027-7. 


R. Vock, A. Dieckmann, S. Ochmann, R. Klein, Fast template matching 
and pose estimation in 3d point clouds, Computers & Graphics 79 (2019) 
36-45. doi:https://doi.org/10.1016/j.cag.2018.12.007. 


H. Huang, S. Wu, D. Cohen-Or, M. Gong, H. Zhang, G. Li, B. Chen, 
L1-medial skeleton of point cloud, ACM Trans. Graph. 32 (2013). 
doi:10.1145/2461912. 2461913. 


C. Wang, Q. Shu, Y. Yang, W. Chen, Quick registration algorithm of 
point clouds using structure feature, Guangxue Xuebao/Acta Optica 
Sinica 38 (2018). doi:10.3788/A0S201838 .0911005. 


23 


